In general, deep learning is defined as a set of machine learning algorithms that try to achieve a high level of abstraction through a combination of various nonlinear transformation techniques, and is a field of machine learning that teaches computers how to think like people do.
A number of researches have been carried out to express data in the form that the computers can understand, for example, pixel information of an image as a column vector, and to apply it to the machine learning. As a result of this effort, a variety of deep learning techniques such as deep neural networks, convolutional neural networks, and recurrent neural networks have been applied to various fields like computer vision, voice recognition, natural language processing, and voice/signal processing, etc., and high performing deep learning networks have been developed.
These deep learning networks are evolving into a large scale model with a deep hierarchy and wide features in order to improve the recognition performance.
In particular, learning of the deep learning network is mainly carried out on servers on-line because of the necessity of large-scale training data and high computing power.
However, it is impossible to perform learning on the servers in a personal mobile device environment where personal data cannot be transmitted to the servers for learning purposes due to privacy concerns, or in environments of a military, a drone, or a ship where the device is often out of the communication network.
Therefore, on-device learning of the deep learning network should be performed in the local device where it is impossible to learn on the servers.
However, the local device performing on-device learning has no room for storage of the training data, and thus it is difficult to perform on-device learning.
In addition, when learning the deep learning network with new training data, if the new training data is different from past training data, the deep learning network gradually forgets what has been learned in the past. As a result, a catastrophic forgetting problem will occur.
In addition to this, if on-device learning is performed on the local device, a lot of computing power will be required, and the learning itself will take up a lot of time.